REPORT 



RESUMES 



ED Oil W RE 000 lie 

THE USE OF STANDARDIZED TESTS IN EVALUATING A METHOD OF 
TEACHING READING. 

BY- HANSON, JOSEPH T. 

PUB BATE FEB 67 

EDRS PRICE MF-$0.09 HC-$0.44 HP. 

DESCRIPTORS- DREADING RESEARCH, DREADING ACHIEVEMENT, READING 
COMPREHENSION, VOCABULARY, SPELLING, PRIMARY GRADES, READING 
IMPROVEMENT, READING INSTRUCTION, ^INSTRUCT lONAL MATERIALS, 
4:S£X DIFFERENCES, DEVALUATION, TEACHING METHODS, PASADENA, 
PHONOVISUAL reading METHOD, NEW YORK CITY 

SEX DIFFERENCES IN READING ACHIEVEMENT AND THE 
EFFECTIVENESS OF THE PHONOVISUAL METHOD OF READING 
INSTRUCT ICW IN GRADES 1 TO 3 WHEN USED AS A SUPPLEMENT TO THE! 
STATE-ADOPTED CO-BASAL READING SERIES WERE STUDIED IN TWO 
ELEMENTARY SCHOOLS IN PASADENA, CALIFCiRNIA. SUBJECTS IN THE 
EXPERIMENTAL SCHOC>L (6C5 IN GRADE 1, 68 IN GRADE 2, 54 IN 
GRADE 3) WERE MATCHED WITH CONTROL SCHCOL SUBJECTS CM THE 
BASIS OF CHRONOLOGICAL AGE, SEX, AND SCHOLASTIC APTITUDE AS 
MEASURED BY THE METROPC»LITAN READINESS TESTS, CALIFORNIA 
SHORT-FORM TEST OF MENTAL MATURITY, AND THE LCRGE-Th|ORNDIKE 
INTELLIGENCE TESTS. EXPERIMENTAL SUBJECTS WERE GIVEN DAILY 
INSTRUCTION BY THE PHONOVISUAL METHOD lOG MINUTES PER WEEK 
WHILE CONTROL SUBJECTS WERE GIVEN AN EQUIVALENT A^^OUNT CF 
INSTRUCTION WITH OTHER SUPPLEMENTARY MATERIALS. ACHIEVEMENT 
WAS TESTED BY THE CALIFORNIA READING TEST AND CALIFORNIA 
ACHIEVEMENT TESTS, READING AND SPELLING. MEAN TEST SCORES CF 
THE EXPERIMENTAL GROUP WERE FOUND TO BE SIGNIFICANTLY HIGHER 
AT THE .01 LEVEL OF ALL TESTS IN ALL GRADES EXCEPT SPELLING 
IN grade 3, WHICH WAS SIGNIFICANT AT THE .05 LEVEL. GIRLS’ 
ACHIEVEMENT WAS FOUND TO BE HIGHER THAN BOYS’ EXCEPT IN GRADE 
3 OF THE EXPERIMENTAL GROUP. THE AUTHOR CONCLUDED THAT THE 
ADDITIONAL PHONICS INSTRUCTION WITH THE PHC^JOVISUAL METHOD 
INCREASED READING AND SPELLING ACHIEVEMENT. THIS PAPER WAS 
READ AT THE ANNUAL MEETING CF THE NATIONAL COUNCIL CM 
MEASUREMENT IN EDUCATION (NEW YORK, FEBRUARY 1967). (LS) 
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An interest in the use of additional structured phonics instruction 
in the teaching of reading prompted the principal and primary teachers at 
the Don Benito Elementa^ry School in Pasadena, California, to request a 
pilot project with the Phonovisual Method (5) and materials to supplement 
the basic method of teaching reading in the Pasadena City Schools. The 
principal and primary grade teachers at Willard Elementary School agreed 
to have their primary grades serve as Control groups. After careful and 
considerable inservice training, the experimental study (4) was initiated 
in the spring of 1963. 
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The purpose of this research was to evaluate the effectiveness of the 
Phonovisual Method of reading instruction in kindergarten and grades 1-3 
when used as a supplement to the State-adopted Co-Basal Reading Series and 
to observe any differences in reading achievement related to sex. The 
main questions to be answered were: 

1. Will pupils achieve equally well in reading without the supple- 
mentary instruction in phonics? 

^Paper read at the 1967 annual meeting of the National Council on 
Measurement in Education, New York, N.Y., February, 1967. 
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2, Will boys achieve as well in reading without the supplementary 
instruction in phonics? 

3. Will girls achieve as well in reading without the supplementary 
instruction in phonics? 

METHOD 

The design of the experiment called for Experimental and Control 
schools with corrparable socio-economic characteristics and pupils of com- 
parable age and abilities. Groups in grades one, two and three in the two 
schools were matched on scholastic aptitude, chronological age and sex. 

The scholastic aptitude measures used were: grade one - Metropolitan Readi- 
ness Tests; grade two - California Short- Form Test of Mental Maturity; 
grade three - Lorge- Thorndike Intelligence Tests , A boy in the Experimental 
school was matched with a boy of the same age, of the same mental ability 
and in the same grade in the Control school. Girls were similarly matched. 
Children were considered to be of the same age if within ± three months 

f 

and as having ’’equal" learning readiness or scholastic ability if Total 
Readiness Scores or IQ’s were identical or did not differ more than the 
Standard Error of Measurement of the test. The groups so selected con- 
sisted of 60 pairs in grade one, 30 boys and 30 girls; 68 pairs in grade 
two, 34 boys and 34 girls; and 54 pairs in grade three, 27 boys and 27 
girls. ^ 

The achievement tests used were: grade one - California Reading Test, 

Lower Primary; grade two - California Achievement Tests, Reading and 
Spelling, Upper Primary; grade three - California Achievement Tests, 

Reading and Spelling, Elementary. 

2 

Descriptive statistics of the matched groups will be found in 
An Evaluation of the Phonovisual Method, Grades 1-3 , op. cit. ( 4 ) 
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Children in grades one, two and three of the Experimental school were 
given daily instruction by the Phonovisual Method, which amounted to 100 
minutes per week. The most essential Phonovisual materials used were, 

(a) The ''Phonovisual Method, (b) The Phonovisual Diagnostic Consonant 
Charts, (c) The Phonovisual Diagnostic Vowel Charts, and (d) flic Phono- 
visual Skill Builders. Children in the Control school also were given an 
equivalent amount of additional instruction in reading based upon organized 
material drawn from the following materials published by the Division of 
Instructional Service, Pasadena City Schools (3): (a) The APPENDIX of 
A Planned Sequence of Learnings in Phonics for Grades 1-3 , (b) Supplemen- 
tary materials listed in Guidelines for the Elementary Reading Program , 

(c) Reading and Literature section of Curriculum Guide for the Elementary 
Schools , and (d) Revised Reading Worksheets and Directions for Grades I, 

II, III . 

In both Experimental and Control schools, the increased instruction 
was in addition to the scheduled reading instruction based upon the 
California Co-Basal Reading Series (1). Since the school day was not 
lengthened in either Control or Experimental school, the added time of 
20 minutes per day was taken from other subjects: approximately five minutes 
daily from Opening Exercises, five minutes daily from Language Arts, and 
ten minutes daily from Social -Studies . The ten minutes taken from Social 
Studies were replaced by social studies content in the regular reading 
periods. 

Teachers in the Control school were told that their school was the 
Control school and that their classes were serving as Control classes. 

They had to be told, of course, about the additional time to be spent on 
reading and planned for the most profitable use of this time. They 
participated in the selection and organization of materials and had 
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additional inservice training. It is believed that all of these factors 
tended to maximize any possible Hawthorne Effect in the Control school 
and, consequently, any greater gain by pupils in the Experimental school 
is even more noteworthy. 



RESULTS 

The performance of Experimental and Control groups on achievement 
tests is reported in Table 1. 

The t-test was used to determine significance of differences between 
mean achievement of the groups (2). It was found that mean test scores of 
pupils in the Experimental groups exceeded the mean test scores of the 
pupils in the Control groups on all tests in all grades. As the data in 
Table 1 indicate, the differences were significant at the .01 level on all 
tests except Spelling in grade three. The hypothesis that pupils would 
achieve equally well in reading without the supplementary instruction in 
phonics had to be rejected. The results of statistical analyses of 
achievement test scores attained by Experimental and Control groups at 
each grade level - one, two and three - in Pasadena’s Phonovisual pilot 
project support the conclusion that the Phonovisual materials and method 
of instruction have a salutary effect in the reading and spelling programs 
of the primary grades. All differences in mean performance between the 
two groups in each grade were in favor of the Experimental group and 
statistically significant. 

In selecting the equivalent ’’grade level” samples with respect to 
scholastic ability and chronological age, equivalent groups of boyS and 
equivalent groups of girls were drawn first. It was possible, therefore, 
to analyze the achievement test data of each sex. The performance of 



Table 1 



Comparison of Test Scores Achieved by Experimental and Control 
in Grades 1, 2 and 3 on the California Achievement Tests 



Grade 1 

Lower Primary Test 


JL 


Vocabulary 

Experimental 


60 


Control 


60 


Comprehension 

Experimental 


60 


Control 


60 



Grade 2 

Upper Primary Test 



Vocabulary 



Experimental 


68 


Control 


68 


Comprehension 




Experimental 


68 


Control 


68 


Spelling 




Experimental 


68 


Control 


68 


Grade 3 

Elementary Tests 




Vocabulary 




Ejqperimental 


54 


Control 


54 


Comprehension 




Experimental 


54 


Control 


54 


Spelling 





Experimental 54 



54 



Mean 



Score 


SD 


SEm 


61.93 


10.13 


1.32 


51.73 


11.57 


1.51 


7.97 


4.22 


.55 


5.10 


3.92 


.51 



37.03 


6.23 


.76 


33.82 


6.39 


.78 


36.15 


9.12 


1.11 


31.65 


10.15 


1.24 


13.81 


4.69 


.57 


11.78 


3.41 


.42 



33.80 


5.42 


.74 


28 .48 


8.13 


1.12 


40.59 


10.43 


1.43 


34.11 


11.51 


1.58 


15.09 


5.14 


.71 


12.94 


5.49 


.75 



Me-Mc SEd 



10.20 1.51 



2.87 .62 



3.21 1.03 



4.50 1.60 



2.03 .71 



5.32 1.29 



6.48 2.15 



Groups 



t 



6.75 



4.63 



3.12 



2.81 



2.86 



4.12 



3.01 



Control 



2.15 



1.03 2.09 
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first and third grade boys and girls on achievement tests used in this 
study are summarized in Table 2. On the basis of mean test scores achieved 
by the respective groups in grade one, the hypothesis that boys would 
achieve as well in reading without the supplementary instruction in phonics 
and the hypothesis that girls would achieve as well in reading without the 
supplementary instruction in phonics had to be rejected. 

The grade one mean test scores of boys in the Experimental group 
exceeded the mean test scores of boys in the Control group on the Vocabu- . 
lary test and on the Comprehension test. The results were similar for the 
girls in both groups. All the differences in the first grade were signifi- 
cant at the .01 level. 

In the third grade, it will be noted that the boys of the Experimental 
group achieved higher mean scores than did the boys in the Control group. 
The difference was significant at the .01 level on the Vocabulary test and 
at the .05 level on Comprehension and Spelling tests. The mean score made 
by the girls in the third grade E5q>erimental group exceeded the mean score 
made by the girls in the Control group on the Vocabulary test. The dif- 
ference was significant at the .05 level. The obtained difference of 4.86 
raw score points between means on the Comprehension test favored the girls 
in the third grade Experimental group. This considerable difference in 
relation to the large standard error of the difference for the small sul>“ 
samples fell between the .10 and the .05 level of significance. 

DISCUSSION 

All the differences between mean scores indicate higher mean achieve- 
ment test scores made by children in the Experimental school. An 
interesting conparison can be made with respect to the tesit performance 
of third grade boys and girls. Although the mean scores of the girls in 
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Performance of First and Third Grade Boys and Girls in Experimental and 
Control Groups on the California Reading and Spelling Tests 

Mean 







N 


Score 


SD 


SEm 


Me~Mc 


SEd 


t 




Grade 1 












* 




Vocabulary 
















Boys 


- Experimental 


30 


59.37 


11.84 


2.20 


8.80 


2.24 


3.93 


Boys 


- Control 


30 


50.57 


12.70 


2.36 








Girls 


- Experimental 


30 


64.50 


7.12 


1.32 


11.60 


2.00 


5.80 


Girls 


- Control 


30 


52.90 


10. 14 


1.89 








Comprehension 
















Boys 


- Experimental 


30 


7.13 


4.23 


.78 


2.56 


.86 


2.98 


Boys 


- Control 


30 


4.57 


3.55 


.66 








Girls 


- Experimental 


30 


8.80 


4.05 


.75 


3.17 


.92 


3.45 


Girls 


- Control 


30 


5.63 


4.20 


.78 










Grade 3 
















Vocabulary 
















Boys 


- Experiifiental 


27 


34.89 


5.59 


1.10 


7.70 


2.12 


3.63 


Boys 


- Control 


27 


27.19 


8.45 


1.66 








Girls 


- Experimental 


27 


32.90 


5.09 


1.00 


3.12 


1.39 


2.24 


Girls 


- Control 


27 


29.78 


7.55 


1.48 








Comprehension 
















Boys 


- Experimental 


27 


41.00 


11.81 


2.32 


8.11 


3.29 


2.47 


Boys 


- Control 


27 


32.89 


11.68 


2.29 








Girls 


- Experimental 


27 


40.19 


8.76 


1.72 


4.86 


2.81 


1.73 


Girls 


- Control 


27 


35.33 


11.20 


2.20 








Spelling 
















Boys 


- Experimental 


27 


14.22 


5.31 


1.04 


3.03 


1.36 


2.23 


Boys 


- Control 


27 


11.19 


3.62 


.71 








Girls 


- Experimental 


27 


15.96 


4.80 


.94 


1.26 


1.48 


.85 


Girls 


- Control 


27 


14.70 


6.38 


1.25 
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the Control group exceeded the mean scores of the boys in the Control 
group “ a familiar difference on reading achievement tests - the mean 
scores achieved in reading by boys in the third grade Experimental group 
not only exceeded the mean scores achieved by the boys in the third grade 
Control group, but also exceeded the mean scores achieved by tlte girls in 
the Ejqperimental group. 



SUMMARY 

The puipose of this research was to evaluate the effectiveness of the 
Phonovisual Method of reading instruction used to supplement the State- 
adopted Co-Basal Reading Series. Analyses of the achievement test results 
support the conclusion that the additional phonics instruction with the 
Phonovisual Method and materials increased reading and spelling achieve- 
ment of the pupils who participated in this study. 
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